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1. Introduction. 



Symmetric functions {E njk (X)} k=l , defined by the Newton inter- 
polation 



plays an important role in the Garsia-Haglund proof of the q, t-Catalan 
conjecture, CD. 

Let Aq/ t -> be the space of symmetric functions of degree n, over 
the field of rational functions Q(q, t), and let V : Aq/ t s — > A^ q - 
be the Garsia-Bergeron operator. 

By studying recursions, Garsia and Haglund show that the co- 
efficient of the elementary symmetric function e n (X) in the im- 
age V(E ntk (X)) of E ntk (X) is equal to the following combinatorial 
summation 



where D nyk is the set of all Dyck paths with initial k North steps fol- 
lowed by an East step. Here area(n) and bounce(n) are two num- 
bers associated with a Dyck path n. It is conjectured in 01, more 
generally, that the VE n ^ k (X) are "Schur positive." 

In [0Q|, using (11.11 ), Can and Loehr prove the q, t-Square conjecture 
of the Loehr and Warrington Q. 

The aim of this article is to understand the functions {E n ^ k [X)^_^ 
better. We prove that the vector subspace generated by the set {E Htk (X) } k 
of the space Ag, (1 of degree n symmetric functions over the field 




(1.1) (V(E n>k (X)),e n (X)) = q area ^ ] t bounc ^\ 
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Q(q), is equal to the subspace generated by 

{s {ktln - k) [X/(l-q)}}) 



n 



Schur functions of hook shape, plethystically evaluated at X/ (l—q). 

In particular, we determine explicitly the transition matrix and its 
inverse from {E rii k(X)} k=1 to {sf k>ln -k^[X/(l — q)]} k=v The entries 
of the matrix turns out to be cocharge Kostka-Foulkes polynomials. 

We find the expansion of E n ^(X) into the Hall-Littlewood ba- 
sis, and as a corollary we recover a closed formula for the cocharge 
Kostka-Foulkes polynomials K\^(q) when A is a hook shape; 



Here, /i is a partition of n whose first column is of height r. 



2.0.1. Notation. A partition /i of 11 G Z >0 , denoted /i h n, is a 
nonincreasing sequence fii > /i 2 > • • • > /-tfc > of numbers such 
that J2 = n - The conjugate partition // = > . . . > ji' s > is 
defined by setting /i- = |{/i r : /i r > 

Par(n,r) denotes the set of all partitions /i h n whose biggest 
part is equal to /^i = r. 

We identify a partition /i with its Ferrers diagram, in French no- 
tation. Thus, if the parts of fi are /ii > fi 2 > ■ ■ ■ > > 0, then 
the corresponding Ferrers diagram have ^ lattice cells in the i th row 
(counting from bottom to up). 

Following Macdonald, jH the arm, leg, coarm and coleg of a lat- 
tice square s are the parameters a M (s), l^s), a'^s) and I'^s) giving 
the number of cells of /i that are respectively strictly EAST, NORTH, 
WEST and SOUTH of s in //. 

Given a partition ji = ji 2 , ■ ■ ■ , H-k), we set 



k 



(n— fc,l fc 



)M (g) = (-l) fc ^(-l)^) 



2. Background. 



(2.1) 
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We also set 

l 2 - 2) 

K(q,t)=Y[(q a ^-t l ^ +1 ) and h'^q, t) = Y[(t l ^-q a ^ +1 ). 

Let F be a field, and let X = {x±,X2, ■■■} be an alphabet (a set 
of indeterminates). The algebra of symmetric functions over F with 
the variable set X is denoted by A F (X). 

If Q C F, it is well known that A F (X) is freely generated by the 
set of power-sum symmetric functions 

{p r (X) : r = 1, 2, ... and, p r (X) = x\ + x\ + ■ ■ ■ }. 

The algebra, A ¥ (X) has a natural grading (by degree). 

Af(X) = 0A«(X), 

n>0 

where Ap(X) is the space of homogenous symmetric functions of 
degree n. 

A basis for the vector space AJJ(X) is give by the set {p^j^hn, 

k k 

(2.3) p^X) = Hp^X), where ^ = 

i=i i=i 

Another basis for Ap(X) is given by the Schur functions {s^X)}^, 
where s^X) is defined as follows. Let 

(2.4) e n (X) = ^2 x h Xi 2 ---x in 

l<h<—<i„ 

be the n'th elementary symmetric function. If fi = Yli=i men 

(2.5) s^X) = det(e M /_i +i (X)) 1 < iij < m , 

where yU- is the i'th part of the conjugate partition /i' = /i^) 
and m> I. 

2.0.2. Plethysm. For the purposes of this section, we represent an 
alphabet X = {xi,x 2 , ■■■} as a formal sum X = J2 x i- Thus, if 
Y = Vi is another alphabet, then 

(2.6) XY = (^XiX^yi) = ^Xiyj = {xij/j}ij>i, 

i,3 
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and 

(2.7) X + Y = (Y t x i ) + (J2vj) = {■'■;■!/,},> , 

i 3 

The formal additive inverse, denoted —X, of an alphabet X = 
Xi is defined so that —X + X = 0. 

In this vein, if Pk(X) = J2 k>1 x\ is a power sum symmetric func- 
tion, we define 

(2.8) Pk [XY] = Pk [X}p k [Y] 

(2.9) p k [X + Y] = p k [X]+p k [Y] 

(2.10) Pk [-X] = -p k [X]. 

This operation is called plethysm. Since A F is freely generated 
by the power sums, the plethysm operator can be extended to the 
other symmetric functions. In fact, using plethysm, one defines the 
following bases for A^ and t y respectively. 

Theorem-Definition 1. (cocharge Hall-Littlewood polynomials) 
There exists a basis {H^(X; q)}^ n for the vector space A^, which 
is uniquely characterized by the properties 

(1) H^X;q)eZ[q]{s x :\> f i}, 

(2) H £ [{l-q)X;q] e Z[q]{s x : A > //}, 

(3) (H tl (X;q),s {n) ) = l. 

Theorem-Definition 2. (Modified Macdonald polynomials) 
There exists a basis {H^X; q, £)} M i-n for the vector space A^^, 
which is uniquely characterized by the properties 

(1) H fl {X;q,t)eZ[q]{s x :X>fi}, 

(2) /y(l - q)X- q,t] E Z[q]{s x : A > //}, 

(3) (H,[X(l-t);q,t},s {n) ) = l. 

It follows from these Theorem-Definitions that 

(2.11) H,(X;0,t) = H^X-t), 

(2.12) %{X;q,t) = H„,(X;t,q). 
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2.0.3. Kostka-Foulkes and Kostka-Macdonald polynomials. Let H^X; q) = 

K Xfl (q)s x , and H^X; q, t) = J2\ K xM, t)sx be, respectively, 
the Schur basis expansions of the Hall-Littlewood and Macdonald 
symmetric functions. The coefficients of the Schur functions are 
called, respectively, the cocharge Kostka-Foulkes polynomials, and 
the modified Kostka-Macdonald polynomials. It is known that K Xlx (q,t), K Xll (q) e 



N[q,t}. 



It follows from equation (12.1 II) and the Schur basis expansions 
that 



(2.13) K XfM (0,t) = K XlM (t). 



2.0.4. Cauchy Identities. Let X = x % be an alphabet, and let 

Q[X] = exp(J2ZiPk(X)/k). Then, 



(2.14) 




oo 



(2.15) 



Q[X] 



X>i»(*)- 



n=0 



If Y 



^2 Hi is another alphabet, then 



(2.16) 




(2.17) 




where h^(q, t) and h'^q, t) are as in (12.21) . 



(2.18) 




2.0.5. Cauchy's q-binomial theorem. Let (z;q)k 
qz) ■ ■ ■ (1 — q k ~ 1 z), and let 

(q;q)k 



(1 - z)(l 



rJ (9; ?)»•(?; ?)*-/ 
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Then, the Cauchy g-binomial theorem states that 
(2.19) 

(z; q) k = q, q"- 1 } = £ Sq® (-l) r [* 

r=0 r=0 L - 



3. Symmetric functions E ntk (X). 

The family {-E njfc (X)}£ =1 of symmetric functions are defined by 
the plethystic identity 



(3.1) 



e n [X 



1 



1 - q 1^(q->q) 



Let < k < r, and let 



(3.2) 



T k+ i, r = (-1)^(-1)V 



i=0 



Proposition 3.1. For fc = 0, n — 1, 



(3.3) s (fc+ljl n- fc -i)[X/(l - g)] = Tfe+i, 



r=fc+l 



E n>r {X) 

(q;q) r 



Proof. Using the Cauchy g-binomial theorem, we see that the coef- 
ficient of (— z) k on the right hand side of (13.11) is 



(3.4) 



n—k 

E 

i=0 



k + % 
k 



n,k+i 



(q;q) 



k+i 
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On the other hand, by the identities (12.161) and (12.18b . 



e n [X 



— ] 



A 



X 



A'=(n-r,l r ) 
n-1 



^S (r+lil n-,-l)[y— ](-z) r (l - Z) 



r=0 



A' 



A" 



A 



A 



] + (-«)(«!»[- ] + S 2 ,in-a[- ]) + • • • + (-^) n S n [- ] 



1-g 



1-g 



l-g J 



Comparing the coefficient of (— z) k gives, for k > 1 
(3.5) 

[ k \ q {q;q) k +i L l - q 1 



E n> k+i r A 1 r A 

s k,i n - k [~i z\ s fe+i,i n-fc - 1 



l-g J 



l-g J 



and 



(3.6) 



A 



l-g J 



We take the alternating sums of the equations (13.5b and (I3.6h to 



get 



By collecting E n>fc (A)'s, and using (|2.19h we obtain 



* + J 



(q;q) 



i+j 



S(fc+i,i«-*-i)[-^/(i - g)] = E T ' 



fc+l,r~ 



r=fc+l 



(q;q)r 



□ 



8 



MAHIR BILEN CAN 



Let S and E be the matrices 

/ 8l n[X/(l-q)] \ 
s 2)ln -i[X/(l-g)] 

V s n [X/(l-q)} J 



and E 



(a}9)i 

-C/ra,2 
(9;9)2 



V (9:9)71/ 



respectively, and let T be the transition matrix from i£ to S, so that 
5 = TE. Then, T is an upper triangular matrix with the k + 1, r'th 
entry 



■fc+l,r 



(-1)* 



For example, when n — 5, 

/111 1 

g (g + l)g (g 2 + g + l)g 



T 



\ 



(g 3 + g 2 + q + 1) g 












g 3 (g 2 + g+l) g 3 (g 4 + g 3 + 2g 2 + g + 1) 
g 6 g 6 (g 3 + g 2 + g + 1) 



\ 



\ 












g 10 


Then, 














( 1 




q- 2 


_g-3 


q~ A 







q- 1 


9+1 

9 3 


9 2 +9+l 
9 5 


9 3 +9 2 +9+l 
9 7 


T- l = 










9 2 +9+l 
9 6 


9 4 +g 3 +2 g 2 +g+l 
9 9 















9 3 +9 2 +g+l 

? 10 




\o 











g- 10 



/ 

Proposition 3.2. T _1 z's (necessarily) upper triangular, and its k + 
l,r'th entry is equal to 

(3.7) (T -1 )fc + i jr = (-l) r -V r(fc+1) T fe+ i, r . 

Proof. Let L be the upper triangular matrix with the k + 1 , r'th entry 

£*+i,r = (-l) r - fc g- r(fe+1) T fc+1 , r forr > k. 
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Clearly, TL is an upper triangular matrix, and the i + 1, j'th entry of 
TL is 



(3.8) 



k=l 



It is straightforward to check that (TL) i+ i ti+ i = 1. We use induction 
on j to prove that for alH + 1 < j, (TL) i+l j = 0. So, we assume 
that for alii + 1 < j, (TL) i+1 j = 0, and we are going prove that for 
alii + 1< j + 1, {TL) i+1>j +l = 0. 

First of all, using the g-binomial identity 



g r_m , form > 0, 



(3.9) 

it is easy to show that 



r 




r — 1 


+ 


r 


- r 


m 




m 




m 


- 1 



(3-10) 7i+i,fc — Ti+i,fc-i 

It follows that 

(3.11) Lfc+ij+i = -q^ h+1) L k+1J + q- k L kJ ^. 

Therefore, 

i+i j'+i 

Ti + \ t kLk,j+i = T i+ i^{—q ^ k+1 ^Lkj + q k Lk-ij~i) 

k=i+l k=i+l 

3+1 3+1 

fc=i+l fc=i+l 

Using (13.101) in the last summation, we have 



3+1 3+1 3+1 

Ti + i t kLkj+i = — <7 k Ti + i^Lk,j + 9 ^ 1 ^Ti + i ! fe_i 

fe=i+l fe=i+l fc=i+l 

i+i 

+ qTi+i.k-iLk-i, 



■J- 
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After rearranging the indices, and using the induction hypotheses, 
the right hand side of the equation simplifies to 0. Therefore, the 
proof is complete. 

□ 

Corollary 3.3. Let A C AJ^ (A) be the n-dimensional subspace 
generated by the set {E nt k(X)}^ =1 , and let B C AJL )(A) be the 
n-dimensional subspace generated by {s^in-* [jz-]}Jc=i- Then, A = 
B. 

Proof. It is clear by Proposition 13.21 that A = B. The dimension 
claim follows from Proposition 14. ll below. □ 

The expression (— l) n p n = X]fc=o( — l) fcs fc+i,i n-fe_1 is the bridge 
between Schur functions of hook type with the power sum symmet- 
ric functions. By the linearity of plethysm we have 

n-l 

(-l)> n [A]/(l-g«) = (-l)> n [A/(l-g)] = ^(-l)Vy^[X/(l- 

k=0 

and therefore 

n-l 

(3.12) (-l)> n = (1 - g") £(-1) Vi.i»-^[*/(l - g)]. 

fe=0 

Corollary 3.4. For a// n > 1, 



1 n n 

(3.13) (~1)X = Er -V^,r. 

Proof. By Proposition 13.11 and (13.121) we get 

(3.14) (-!)>„ = (l -<?»)£ £ Wt^- 

k=0r=k+l ^ q)r 

By rearranging the summations and using the Cauchy's g-binomial 
theorem once more, we finish the proof. □ 
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4. Hall-Littlewood expansion. 
Proposition 4.1. For k — 1, n, 

,, ,.. ru,j ,.. ru .„./ ( ,(.):-n> 



froo/ Let K = (l — t)(l — z). Then, by the Cauchy identity (l2T71> . 
we have 

The left hand side of the equation (14.11) is independent of the vari- 
able t. Since h^q, 0) 7^ 0, and since h'^q, 0) 7^ 0, we are allowed 
to make the substitution t = on both sides of the equation. 

Note that 

h^q,0)h(^)0) = Y[q a " is) J] (-e (s)+1 ) II (l-? Ms)+1 ) 

, _ 1 _ „a M (s)+l 

(4.3) = ( -,)»n9 2a ' w n ^ZSPFT 

(4.4) = (-g) n JJg 2a " (s) j [ (1 - g-^W-i) 

se^,/ M (s)=0 

(4.5) = (-g)V n(//) JJ (1-g- ^- 1 ). 

s€/i,i M (s)=0 

The equality (1431) follows from j2J}- 

Using the Schur expansion H^(X;q,t) = J2\^x^(q^) s x, we 
see that the plethystic substitution X — > (1 — z), followed by the 
evaluation at t = is the same as the evaluation H^X, q, 0) at 
t = 0, followed by the plethystic substitution X — > (I — z). Also, 
by Corollary 3.5.20 of H, we know that 



H^[l - z; q,t] = Ql-zBp], where = 



1 — n^ 1 



1 - q 
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Therefore, 

(4.6) #„[l-s;g,0] = fi[-^]|t= 

(4.7) = fi[-z(l + g + --- + g^- 1 )] 

(4.8) = n^ 1 -^) 

i=0 

(4.9) = (z;g) m . 
It follows from dZTTb and (12121) that 

(4.10) ^(X;g,0) = ^(X;g). 

By combining (l4~Ti (1431) , (14~9l ) and (14.101 ), we get 
(4.11) 

By comparing the coefficient of (z; g)^ in (14.1 II) . we find that 
(4.12) 

E n , k {x) ^ g) 

Hence, the proof is complete. 

□ 

Lemma 4.2. Le? X\- n be a partition ofn. Then, 

Al (i-9)(i-*) J £ Ms.t)hV(s,«) 

Proof. This follows from Theorem 1.3 of Q. □ 
Corollary 4.3. Lef \\- nbe a partition. Then, 

X . ^K xl ^(q)H^{X;q) ^ K x ^(q)H^(X;q) 



^2 A Ay^;^^^;g; = ^ 



Proo/ It follows from (|2T2h and (12131) that Xy M (g, 0) = Ky^i (0, g) 
Since, Mg,0)/^M) = (-?) V" 00 IL^)^ 1 - 
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q-aM- 1 ), and 5^X^,0) = H^(X;q), the proof follows from 
Lemma E21 □ 

Theorem 4.4. Let 1 < k < r < n, and let fj, E Par(n, r). Then, 

Tk,r = -^(n-fc+Llfc- 1 )//^)- 

Proof. Recall that 
where 

k 

r 



T k+ l, r = (-1)^(-1)V 



i=0 



Therefore, by Corollary 14.31 and Proposition 14. 1 1 we have 

E K^ n _ k+l l k-i ) ^{q)H^{X] q) r X 1 

ITT 7 ; IT ~~ s k,l n ~ k 



^(g,0)/iV(g,0) ' 1-9 

r=k ^^ParKr)^^' )^^ ) 

Tk, r Hfj,'(X;q) 



The theorem follows from comparison of the coefficients of ff^/ (X;q). 

□ 
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